fix(agent): classify LLM provider 400s as USER with an actionable detail by ionut-mihalache-uipath · Pull Request #1072 · UiPath/uipath-langchain-python

ionut-mihalache-uipath · 2026-09-07T20:27:16Z

Problem

An LLM gateway 400 fell into the unclassified branch of _classify, so it surfaced as HTTP_ERROR / UNKNOWN with detail set to the HTTP reason phrase — the two words "Bad Request".

That is the single largest contributor to the fleet's two-word failure messages (PC-5002), and it buries causes the customer can actually act on. The motivating case: a job that failed 192/192 runs on gpt-4.1-mini-e2e-custom because max_tokens=65535 exceeded the model's 32768 completion-token limit — a value Agent Builder itself had written into the model settings. The customer saw "Bad Request", categorized Unknown.

Change

400 now maps to LLM_PROVIDER_BAD_REQUEST / UiPathErrorCategory.USER, with a title of "LLM provider rejected the request".

For detail, precedence is:

The gateway's ProblemDetails detail — first-party UiPath text, and more specific, so it still wins.
Otherwise a canned message pointing at the agent's model settings (output-token limit, temperature, effort) and noting that the provider's own message is on the LLM call span.

The provider body is deliberately not read out into the error. It may carry customer PII, and it is already recorded on the LLM call span, which is tenant-scoped. USER is not in _SHOULD_WRAP_CATEGORIES, so the canned detail has to stand on its own — tests pin that it isn't prefixed with the generic wrapper.

_classify now returns a fourth element, fallback_detail. None means "fall back to the HTTP reason phrase", so only statuses whose cause we cannot name are left with it.

What is deliberately not in scope

404 stays UNKNOWN. Every LLM-gateway 404 observed in prod over 30 days was a missing or unreachable model deployment — BYO relay not connected, Azure DeploymentNotFound, a retired Bedrock model — which is Deployment, not User. It needs its own decision on its own evidence rather than being folded into this change. test_unclassified_4xx_remains_unknown documents that and covers 404/408/413/422/429.

Tests

400 → USER + canned detail across four body shapes: vendor envelope, Bedrock envelope, raw edge HTML, empty body.
The provider body is never quoted back — asserted over detail, str(error) and repr(error).
A gateway ProblemDetails detail takes precedence over the canned text.
The canned detail is not wrapped in the generic "An unexpected error occurred" prefix.
End-to-end through the LLM node for both the new client (UiPathAPIError) and the legacy raw openai.BadRequestError path.
Unclassified 4xx statuses stay UNKNOWN.

uv run pytest tests/agent/test_llm.py tests/agent/react/test_llm_node.py — 54 passed. ruff check, ruff format, the httpx-client AST linter, and mypy all clean.

Drive-by

Fixed a stale file reference in the develop-agent-module skill: exceptions/licensing.py → exceptions/llm.py.

🤖 Generated with Claude Code

https://claude.ai/code/session_01UZ5LHC1Xhp1zBRfYnYENwv

Copilot

🟢 Approval recommended

The change is well-scoped, avoids provider-body leakage by construction, and is backed by targeted unit + node-level tests that pin the new 400 classification and fallback-detail behavior.

Pull request overview

This PR improves how LLM-gateway HTTP 400 responses are mapped into AgentRuntimeError so customers see an actionable, provider-safe message instead of the unhelpful HTTP reason phrase (“Bad Request”), while keeping other 4xx statuses classified as UNKNOWN per the stated scope.

Changes:

Classifies HTTP 400 as LLM_PROVIDER_BAD_REQUEST with UiPathErrorCategory.USER and a stable canned detail when the gateway provides no ProblemDetails detail.
Extends _classify() to return a fallback_detail so only explicitly-handled statuses avoid reason-phrase fallback.
Adds/updates tests to pin 400 behavior (including “no provider body leakage”) and to ensure other 4xx remain UNKNOWN; updates the agent module skill doc reference.

File summaries

File	Description
`src/uipath_langchain/agent/exceptions/llm.py`	Adds 400 classification with provider-safe fallback detail and threads `fallback_detail` through `raise_for_provider_http_error()`.
`src/uipath_langchain/agent/exceptions/exceptions.py`	Introduces `AgentRuntimeErrorCode.LLM_PROVIDER_BAD_REQUEST`.
`tests/agent/test_llm.py`	Expands contract tests for 400 mapping, detail precedence, and “no provider body quoted back”; pins unclassified 4xx behavior.
`tests/agent/react/test_llm_node.py`	Adds end-to-end node-level assertions for 400 mapping for both new client and legacy OpenAI exception paths.
`.claude/skills/develop-agent-module/SKILL.md`	Fixes the referenced exceptions module path for LLM provider errors.

Review details

Files reviewed: 5/5 changed files
Comments generated: 0
Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

An LLM gateway 400 was falling into the unclassified branch of `_classify`, so it reached telemetry as HTTP_ERROR / UNKNOWN with `detail` set to the HTTP reason phrase — the two words "Bad Request". That is the largest slice of the fleet's two-word failure messages (PC-5002), and it hides causes the customer can actually fix, such as the `max_tokens=65535` that Agent Builder itself wrote into the model settings. 400 now maps to LLM_PROVIDER_BAD_REQUEST / USER. Where the gateway supplies a first-party ProblemDetails `detail`, that still wins; otherwise the error carries a canned message pointing at the agent's model settings. The provider body is deliberately not read out — it may carry customer PII and is already recorded on the tenant-scoped LLM call span. 404 is left in UNKNOWN on purpose: every LLM-gateway 404 seen in prod over 30 days was a missing or unreachable deployment (BYO relay not connected, Azure DeploymentNotFound, a retired Bedrock model), which is Deployment rather than User, and deserves its own decision. Also fixes a stale file reference in the develop-agent-module skill (`exceptions/licensing.py` → `exceptions/llm.py`). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UZ5LHC1Xhp1zBRfYnYENwv

andreiancuta-uipath · 2026-09-17T14:25:46Z


+    @staticmethod
+    def _http_400() -> httpx.Response:
+        """The 400 from job 1fab7e97-...: max_tokens written by Agent Builder."""


Don't reference job keys in code

Dropped. The docstring is now "A gateway 400 as seen in prod: max_tokens above the model's limit." — dc48d7f.

There are still other comments referencing the same job.

Missed one — tests/agent/test_llm.py:248 had the same job key in the _MAX_TOKENS_BODY comment. Removed in 8570485; no 1fab7e97 anywhere in the repo now.

Review follow-ups on #1072: - _classify now reads the gateway's ProblemDetails detail itself and returns the final detail, so the gateway-wins-over-canned precedence no longer leaks into raise_for_provider_http_error. None still means "fall back to the HTTP reason phrase". - Trim the comment on _BAD_REQUEST_DETAIL to the part that carries motivation -- the provider body is not read out because it may carry customer PII -- and move it to the 400 branch it explains. - Drop the job key from the _http_400 fixture docstring. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FcQT9jLym8DrdpTamH38Yy

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FcQT9jLym8DrdpTamH38Yy

Sonar flagged _classify at 18 against a limit of 15: applying the gateway-detail precedence inside every branch repeated a conditional per status. The precedence now applies once. _status_verdict decides code, category, title and the detail this mapping names, _forbidden_verdict holds the 403 body check that was the nested branch, and _classify overlays the gateway's ProblemDetails detail on the result. Complexity per function is now 5, 4 and 2. Behavior is unchanged -- gateway detail still wins over both the canned 400 text and the 404 signature details. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FcQT9jLym8DrdpTamH38Yy

sonarqubecloud · 2026-09-17T15:34:44Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
95.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

Copilot AI lite review requested due to automatic review settings September 7, 2026 20:27

Copilot started reviewing on behalf of ionut-mihalache-uipath September 7, 2026 20:27 View session

Copilot AI reviewed Sep 7, 2026

View reviewed changes

ionut-mihalache-uipath force-pushed the fix/pc-5002-llm-400-user-category branch 3 times, most recently from 27cdc1e to 49fddca Compare September 16, 2026 11:15

ionut-mihalache-uipath force-pushed the fix/pc-5002-llm-400-user-category branch from 49fddca to f3c8d7e Compare September 17, 2026 12:45

andreiancuta-uipath reviewed Sep 17, 2026

View reviewed changes

andreiancuta-uipath approved these changes Sep 17, 2026

View reviewed changes

ionut-mihalache-uipath and others added 2 commits September 17, 2026 17:57

test(agent): drop the remaining job key from the 400 fixture comment

8570485

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FcQT9jLym8DrdpTamH38Yy

Navigation Menu

fix(agent): classify LLM provider 400s as USER with an actionable detail - #1072

fix(agent): classify LLM provider 400s as USER with an actionable detail#1072
ionut-mihalache-uipath wants to merge 4 commits into
mainfrom
fix/pc-5002-llm-400-user-category

ionut-mihalache-uipath commented Sep 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

andreiancuta-uipath Sep 17, 2026

Uh oh!

ionut-mihalache-uipath Sep 17, 2026

Uh oh!

andreiancuta-uipath Sep 17, 2026

Uh oh!

ionut-mihalache-uipath Sep 17, 2026

Uh oh!

Uh oh!

Uh oh!

sonarqubecloud Bot commented Sep 17, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

ionut-mihalache-uipath commented Sep 7, 2026

Problem

Change

What is deliberately not in scope

Tests

Drive-by

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟢 Approval recommended

Uh oh!

andreiancuta-uipath Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

ionut-mihalache-uipath Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

andreiancuta-uipath Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

ionut-mihalache-uipath Sep 17, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

sonarqubecloud Bot commented Sep 17, 2026

Quality Gate passed

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants